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ABSTRACT 

Assessment of learning has become very importantfor government, universities and accrediting agencies. In this ar¬ 
ticle, two variables are examined, leadership and teamwork, in the context of a survey used by one mid-south univer¬ 
sityfor assessment purposes. This survey demonstrates the problems that arise when the sequential steps ofthe research 
design process are notfollowed. I also offer suggestions on how to avoid mistakes during the design process for assess¬ 


ment oflearning. 

INTRODUCTION 

Universities today are increasingly accountable to many 
stakeholders. The government, accrediting agencies, and 
parents expect assurance that students are learning what 
they need in college to be successful in the workforce 
(Liu, 2011). Assessment of learning in higher education 
provides information to help universities focus in on 
problems such as teaching quality and student learning 
outcomes (Fletcher, et. al., (2012). Assessment also pro¬ 
vides information to outside organizations on specific ar¬ 
eas of interest. Suppose companies require good written 
communication skills. Students graduating with this skill 
will be available to fill the employment needs of the or¬ 
ganization. Sponsoring companies benefit through avail¬ 
able labor, universities benefit by placing more graduates 
in jobs, and students benefit because they learn the skills 
necessary to be successful on the job. Two important skills 
organizations request of MBA graduates is leadership and 
teamwork. 

As student learning outcomes become even more impor¬ 
tant in the future, we will see more committees assigned 
the task of measurement design to assess skills such as 
teamwork and leadership. Rather than going to experts, 
this goal may be given to committees of appointed mem¬ 
bers with little or no knowledge of the variables to be 
measured. Additionally, they may be uncomfortable set¬ 
ting standards and/or analyzing data. With intensified 
teaching, service and publication demands, assessment is 
sometimes seen as a project to be conducted for the sake of 
accreditation. The result is the measurement of the wrong 
variables and the collection of useless data. 

The purpose of this paper is twofold. First, two variables 
are examined, leadership and teamwork, in the context 
of a survey used by one mid-south university for assess¬ 


ment purposes. This survey demonstrates the problems 
that arise when research designers do not follow the prop¬ 
er methodology. Years of collecting data can all be for 
naught. Second, I will offer suggestions on ways to avoid 
this situation, hopefully helping faculty with techniques 
to improve their own assessment techniques. 

RESEARCH DESIGN 

The seven steps identified in the research design process 
are as follows: 

1. Identify the research problem 

2. Conduct a review of the research 

3. Specify the purpose of the research 

4. Determine the problem and develop the hypothesis 

5. Data collection 

6. Analyzing and interpreting the data 

7. Reporting and evaluating the results. 

The first step of the research design process is to identify 
the problem. The question in our case was “do our MBA 
students graduate with leadership and teamwork skills”? 
A committee was appointed by the college dean. Of the 
faculty serving on the committee, only one person was 
teaching management, a junior faculty with a degree in 
production. 

While putting so much stress on a junior faculty was 
unfair, it was also unwise. Although working hard 
to complete his task in a timely manner, minimal re¬ 
search was conducted on the theoretical basis for the 
items. To demonstrate the lack of face validity, a brief 
literature review of leadership and teamwork follows. 
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Leadership 

Being able to influence others to perform is the key to 
leadership. Two major types of leaders are discussed in 
management literature, transaction and transforma¬ 
tional. According to the lead researchers on this topic, 
transformational leadership processes may align follow¬ 
ers’ work-oriented values with those of the greater group 
or organization (Bass et. ah, 1987; Burns, 1978; Conger 
& Kanungo, 1988). Transactional leaders perform more 
often in the background of the company. They are con¬ 
cerned with day-to-day operations and maintaining status 
quo. They use legitimate, coercive, and reward power more 
than other types. Transformational leaders are at the cen¬ 
ter of the organization, mostly leading through expert and 
reference power. They look forward to the future of the 
organization and develop a vision that others also follow. 
Transformational leaders possess charisma and are able 
to simply be supportive rather than directive when the 
situation allows (Hersey & Blanchard, 1984). There is a 
religious-like motivation that the leader is able to instill in 
the employees, whereas, the transactional leader appeals 
to employees’ self-interests rather than raise the levels of 
morality and motivation (Burns 1978). 

Leadership studies often focus on a top ranking corporate 
officer. One will find leaders in all areas and positions. 
They may be formal leaders, such as those appointed by 
the organization. But they can also be found in informal 
settings, such as friendship or interest groups. Leadership 
qualities can trickle down through layers of the hierarchy. 
In fact, the relationship between the leadership style of 
the person in charge and the operating employees may be 
irrelevant. It may be the immediate supervisor’s leadership 
behavior that influences success rather than the person in 
charge of the project (Bass and Avolio, 1994). 

One example of a pre-existing scale that might be used in 
place of developing a scale is the Peer Leadership scale de¬ 
veloped by Taylor and Bowers in 1972 (Cook, Hepforth, 
Wall, & Warr, 1981). This instrument has been cited and 
validated repeatedly in leadership literature. It consists of 
an 11 item scale rated on a continuum scale from 1 to 5. 
The dimensions of the scale are support, goal emphasis, 
work facilitation, and interaction facilitation. The next 
section of this paper examines the second dimension of 
interest, teamwork. 

Teamwork 

The definition of teams originated in the 1980’s. While 
the terms “work team” and “work group” are sometimes 
used interchangeably (Hackman, 1990), especially theo¬ 
retically, they are very different variables (Katzenbach 
& Smith, 1993). A team has a purpose outside itself. 


Members gain their identities from the purpose of the 
team and their commitment to the goal, while they are 
also accountable for the task (Rowland, 1989). A group, 
in contrast, serves a common purpose usually relating to 
wider society. The National Association of Professional 
Women (NAPW) is a group. Their mission is to promote 
awareness, networking, and career building for women. 
However, they do not all share the same goal outside of 
the group’s activities. One may be a member for the social 
benefit, while another may join to find business opportu¬ 
nities. Group members are not accountable for the success 
of NAPW in a way that team members are accountable 
for the service or product they produce. Thus, group mem¬ 
bers show less commitment to the goal than team mem¬ 
bers (Rowland, 1989). 

In 1993, Katzenbach and Smith published an article in 
Harvard Business Review in which they described the dif¬ 
ference between groups and teams (See Table 1). 


Table 1 

Group vs. Team Characteristics 

Group 

Team 

Strong leader 

Shared leadership 

Individual accountability 

Mutual accountability 

Same purpose as 
organization 

Team defines specific 
team purpose 

Individual work products 

Collective work products 

Meeting efficiency 

Open-ended discussion 
and problem-solving 

Measures effectiveness 
indirectly 

Measures effectiveness 
directly by results 

Discusses, decides, 
delegates 

Discusses, decides, and 
does real work together 


Assessment Instrument 

I recently served on a sub-committee for an assurance of 
learning committee. My task was to analyze data that had 
been collected from seniors in a graduate MBA class. The 
analysis intent was to prove graduates’ success with team 
and leadership skills. Immediately upon examination of 
the survey items, I began to doubt the validity of the in¬ 
strument. After three years of collecting data, the surveys 
being analyzed held no face validity (See appendix for 
items). The variables and outcomes of the analysis are fur¬ 
ther explored in the next section of this article. 
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ANALYSIS 

To assess leadership and teamwork, we surveyed students 
from senior graduate classes from 2009 through 2011. 
(See Appendix for Survey Items). The instrument was pre¬ 
sented to students at the end of the semester, after partici¬ 
pation in four to five member teams during one semester. 
Each student was asked to rate themselves and each mem¬ 
ber of their team with regards to: attendance and partici¬ 
pation in meetings, the quality and quantity of members’ 
work, the professionalism of members, and the resilience, 
or positive attitude, to the project demonstrated by mem¬ 
bers. See Table 2 for the descriptive statistics of the data. 
The range of all items is from one to three with the excep¬ 
tion of Quantity. No student reported lower than two for 
this question. All averages are above 2.75 on a scale of one 
to three. Reliability was estimated using Cronbach’s alpha 
at .73. 


Table 2 

Descriptive Statistics 


Min. 

Max. 

Mean 

Std. 

Deviation 

Attendance 

1.00 

3.00 

2.8924 

.32478 

Quality 

2.00 

3.00 

2.9058 

.29272 

Quantity 

1.00 

3.00 

2.6547 

.52166 

Resilience 

1.00 

3.00 

2.8700 

.35022 

Professionalism 

1.00 

3.00 

2.9417 

.25329 


The percentage response for each participant is as follows. 
The scores above an average of 2.7 made up the majority 
of responses at 82.1 %. The second group at 2.6 made up 
10.3%, while only 3.1% scored their team members at 
2.40. Data collection resulted in 223 usable responses. To 
analyze the data we used SPSS 20. 

Two items were used in our study to measure leadership. 
The first item was named resilience and asked about coop¬ 


eration and remaining positive during disagreements. The 
second item was called professionalism and asked about 
the respect for members of the team. Neither of these 
items asked about influence. There was no face validity 
that these questions measured leadership skills in our stu¬ 
dents. 

The items used to measure teamwork in our study are at¬ 
tendance and punctuality, work quality, and work quan¬ 
tity. According to Table 1, these items are related to effi¬ 
ciency and individual accountability. Hence they measure 
group work, not team work. Our survey lacked face valid¬ 
ity and required a deeper probe into whether the results 
were really valid. 

We conducted a preliminary examination of the team¬ 
work-leadership scale. Factor analysis revealed that all 
factors loaded into one factor within an acceptable range, 
indicating one variable, instead of two (Table 3,4 and 5). 
But this did not tell us which variable was being measured. 


Table 3 

COMMUNALITIES 


Initial 

Extraction 

Attendance 

1.000 

.559 

Quality 

1.000 

.492 

Quantity 

1.000 

.486 

Resilience 

1.000 

.434 

Professionalism 

1.000 

.596 

Extraction Method: Principal Component Analysis. 


Correlations for all scale items are in Table 6. As evi¬ 
denced below, all items correlate significantly with each 
other in congruence with the factor analysis. 


Table 4 

Total Variance Explained 

Component 

Initial Eigenvalues 

Extraction Sums of Squared Loadings 

Total 

% of Variance 

Cumulative % 

Total 

% of Variance 

Cumulative % 

1 

2.567 

51.339 

51.339 

2.567 

51.339 

51.339 

2 

.756 

15.129 

66.468 




3 

.708 

14.152 

80.620 




4 

.550 

10.999 

91.619 




5 

.419 

8.381 

100.000 




Extraction Method: Principal Component Analysis. 
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Table 5 

Component Matrix 4 


Component 

1 

Attendance 

.748 

Quality 

.702 

Quantity 

.697 

Resilience 

.659 

Professionalism 

.772 

Principal Component Analysis. 

a. 1 components extracted. 


CONCLUSION 

When conducting in-house research it is important to fol¬ 
low the sequential steps of the correct methodology. Pro¬ 
fessors follow the steps in writing their dissertations or 
thesis. When teaching research methodology, professors 
deduct points when students skip steps or do not follow 
the sequential steps. When conducting and publishing re¬ 
search, professors are held to very high ethical standards 
of reporting: thus they follow the proper steps. Then why 


is it that committees developing measures of learning out¬ 
comes do not always practice what they teach? 

The survey investigated herein did little to measure either 
leadership or teamwork. Three years ofcollectingdatawere 
wasted because the committee did not approach this task 
with the same perseverance of a personal research project. 
Perhaps faculty perceives assessment as just an added task 
to be completed as quickly as possible. However, by not 
following the proper design methodology, the results and 
evaluations are meaningless. By seeing our mistakes, fac¬ 
ulty may be able to avoid them at their schools. Following 
is a list of suggestions to improve the process of assessing 
leadership and teamwork. 

Suggestion 1: Contact experts to identify the variables. 
It is also important to place experts on the committee. 
Experts were not contacted regarding the variables being 
studied herein. 

Suggestion 2: Conduct a thorough review of the litera¬ 
ture to identify any existing scales. Using an existing scale 
that has been validated can reduce the time and cost of 
in-house development. There was little review of the lit¬ 
erature in developing questions for the survey. 


Table 6 

Correlations 


Attendance 

Quality 

Quantity 

Resilience 

Professionalism 

Attendance 

Pearson Correlation 

1 

.319** 

44‘4-** 

.352** 

.526** 

Sig. (2-tailed) 


.000 

.000 

.000 

.000 







Quality 

Pearson Correlation 

.319** 

1 

. 435 ** 

.363** 

.412** 

Sig. (2-tailed) 

.000 


.000 

.000 

.000 







Quantity 

Pearson Correlation 


.435** 

1 

.271** 

.358** 

Sig. (2-tailed) 

.000 

.000 


.000 

.000 







Resilience 

Pearson Correlation 

.352** 

.363** 

.271** 

1 

.422** 

Sig. (2-tailed) 

.000 

.000 

.000 


.000 







Professionalism 

Pearson Correlation 

.526** 

.412** 

.358** 

.422** 

1 

Sig. (2-tailed) 

.000 

.000 

.000 

.000 








**. Correlation is significant at the 0.01 level (2-tailed). 
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Suggestion 3: Keep the high ethical standards used in 
published research. The purpose of this research was to 
satisfy accreditation standards, not to investigate a real- 
life problem. Because the results were not to be published, 
standard research standards were not applied. 

Suggestion 4. Develop the hypothesis based on the litera¬ 
ture review and the experts involved in the process. There 
was no hypothesis, so it was difficult to know what we 
were looking for. 

Suggestion 5: Conduct a pilot study before disseminat¬ 
ing to students. No pilot study was conducted. Data was 
collected with an improper survey for three years. 

Considering the statistical results of the analysis, the 
measurement tool designed to assess whether graduat¬ 
ing MBA students were leaders and worked well in teams 
is not a valid instrument. The recommendation for the 
graduate committee was to develop a different scale for 
the leadership and teamwork constructs. The work of 
three years of meetings and collecting data were a waste of 
the faculties’ time and energy. By examining the mistakes 
made by our committee, we hope to help others avoid the 
same problems. 

One question that should be pursued is why the students 
are rating each other so high. Is it because in graduate 
school all students are more invested in their education? 
Are they really this good? Another proposal might be the 
question of their generosity when rating. Are they more 
forgiving than undergraduates? Since many of our MBAs 
are already working, they may have experienced social 
loafing in the workplace and already be conditioned to ac¬ 
cept this as a reality. Therefore, their expectations may be 
lower. Another problem with this analysis is the lack of a 
social desirability scale to determine if they are rating the 
way they think is socially acceptable or are they just giving 
everyone the highest score across the board. 
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APPENDIX 

Teamwork & Leadership Evaluation Form 

TEAMWORK: Attendance and Punctuality 

3. Attended all meetings; never arrived late nor left early 

2. Attended almost all meetings; arrived late or left early sometimes 

1. Attended a few or no meetings; often arrived late or left early 

TEAMWORK: Work Quality 

3. Brought good or exceptional ideas to make the team project better 

2. Brought some ideas that can be used for the team project 

1. Brought ideas that do not help the team project 

TEAMWORK: Work Quantity 

3. Performed more work than most team members 

2. Performed the same amount of work as most team members 

1. Performed less work than most team members 

LEADERSHIP: Resilience 

3. Was very positive and productive when having disagreements or discussing changes 

2. Was positive and productive when having disagreements or discussing changes 

1. Was negative and less productive when having disagreements or discussing changes 

LEADERSHIP: Professionalism 

3. Listened and respected other team members’ ideas 

2. Listened and attempted to understand other team members’ ideas 

1. Was not willing to listen or to understand other team members’ ideas 
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